Universal Access To All Knowledge
Home Donate | Store | Blog | FAQ | Jobs | Volunteer Positions | Contact | Bios | Forums | Projects | Terms, Privacy, & Copyright
Search: Advanced Search
Anonymous User (login or join us)
Upload

Reply to this post | Go Back
View Post [edit]

Poster: please block it Date: Nov 2, 2008 9:53am
Forum: web Subject: robot block overriden

Yesterday I asked my site 'www.derstuttgarter.de' to be blocked. I inserted a robots.txt. Some time later a block messaged was displayed. Today the archived site is available again and not blocked anymore. How can I ensure that my data will be unavailable and my robots.txt will be respected.

Reply to this post
Reply [edit]

Poster: kustota Date: Nov 2, 2008 6:35pm
Forum: web Subject: Re: robot block overriden

if you want your site to be blocked, robots.txt must be found there:

www.derstuttgarter.de/robots.txt

there is no such file on your server. so there is no block.
more info:
http://www.archive.org/about/exclude.php

Reply to this post
Reply [edit]

Poster: please block it Date: Nov 2, 2008 10:07pm
Forum: web Subject: Re: robot block overriden

Thanks, I did not understand the role of automatic forwarding. I however changed it. So I will see what happens. As I changed forwarding in the meantime.

Terms of Use (10 Mar 2001)